Searching for Explanatory Web Pages using Query Expansion

When one tries to use the Web as a dictionary or encyclopedia, entering some single term into a search engine, the highly-ranked pages in the result usually contain many irrelevant or useless sites. The problem is that single-term queries do not contain enough information to specify exactly which sort of pages the user wants. For the analogous problem in TREC, Buckley et al. (1996) have proposed query expansion, also known as pseudo-feedback or two-stage retrieval. In this method the top n documents returned by an initial retrieval are added to the query, which is then used for a second retrieval. This paper contributes, first, new normalization techniques for query expansion, and second, a new “local relevance density” metric which complements the vector product metric for computing similarity between an expanded query and a document. Both of these techniques are shown to be useful for single-term queries in Japanese over the World Wide Web.